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Abstract:  The  PI  (P.  S.  Krishnaprasad)  and  co-PI  (Andrea  Cavagna,  sub-awardee  at  the  ISC- 
CNR,  Rome,  Italy)  joined  forces  in  a  program  of  experimental,  theoretical  and  computational 
investigation  to  uncover  the  fundamental  principles  that  govern  collective  phenomena  in  three 
dimensions  (3D).  The  investigations  considered  phenomena  at  two  ends  of  length  scale:  (a)  Birds 
-  flocking  in  a  species  of  starling;  and  (b)  Insects  -  swanning  in  a  species  of  midge.  A  primary 
objective  of  this  project  was  to  uncover  order  and  structure  in  such  biological  collectives,  model 
the  observed  behavior,  extract  control  laws  at  the  individual  level  that  govern  collective 
behavior,  and  seek  technological  benefits  (in  applications  to  collective  robotics)  from  this 
research.  The  experimental  program  led  by  Cavagna  in  Italy  yielded  new  observations  of 
starlings  and  midges  in  natural  settings  with  higher  temporal  resolution  than  achieved  before.  (A 
prominent  earlier  experimental  investigation  supported  by  the  European  Union  and  also  led  by 
Andrea  Cavagna  had  generated  very  interesting  insights  based  on  data  obtained  at  coarse 
resolution.)  The  new  data  was  analyzed  by  the  Rome  group  via  a  series  of  computer  vision  based 
algorithms  (segmentation,  static  matching,  dynamic  matching,  static  3D-rconstruction,  cluster 
analysis,  stereo  tracking  -  3D  reconstruction),  and  theoretical  principles  shaped  by  statistical 
physics.  The  theoretical  program  in  Maryland  yielded  new  results  on  certain  key,  network  motifs 
(based  on  motion  camouflage  pursuit  or  parallel  navigation,  and  constant  bearing  pursuit)  to 
serve  as  fundamental  building  block  models  of  complex  collective  behavior.  The  results  in  this 
direction  include  stability  classification  of  dynamical  interactions,  extensions  to  3D,  symmetry 
reduction  and  phase  portraits,  top-down  analysis  based  on  configuration  space  methods, 
variational  principles  and  robotic  demonstrations.  New  control-theoretic  algorithms  for 
smoothing  and  curvature  law  extraction  for  trajectories  were  tested  on  starling  data  provided  by 
the  Co-PI.  A  number  of  archival  publications  in  peer-reviewed  high-impact  journals,  theses,  and 
preprints  as  well  as  software  have  been  created  as  a  result  of  this  project.  Further  analyses  and 
development  of  archival  publications  continue. 

Materials  and  Methods:  There  are  two  different  traditions  of  intellectual  inquiry  that  have 
guided  the  progress  made  in  this  project.  The  co-PI  Andrea  Cavagna,  as  a  physicist  rooted  in 


condensed  matter  theory  came  to  the  problem  of  collective  behavior  from  a  statistical  mechanics 
perspective.  Further  he  and  his  team  developed  from  the  ground  up  a  strong  set  of  skills  in  visual 
observations  using  carefully  synchronized  camera  networks  to  resolve  large  flocks  of  birds  as 
they  moved  in  the  skies  over  Rome  to  obtain  useful  quantitative  information  on  trajectories.  The 
associated  methods  (and  results)  are  described  in  detail  in  the  Section  beginning  on  page  10 
about  Experimental  Data  Gathering  and  Theoretical  Analysis.  The  approach  adopted  in  the 
efforts  at  Maryland  is  strongly  shaped  by  the  advances  in  nonlinear  and  geometric  control  theory 
in  which  the  PI  has  made  significant  prior  contributions.  Since  the  beginning  of  2000  these 
methods  were  introduced  into  the  subject  of  collective  behavior  by  the  PI,  with  support  from 
AFOSR,  ARO  and  ONR.  During  the  course  of  this  AFOSR  project  (2010  -  2014)  significant 
new  conceptual  advances  were  made  by  taking  a  geometric  approach  to  the  analysis  and 
synthesis  of  collective  behavior.  First,  the  self-steering  particle  model  of  an  agent  in  a  collective, 
introduced  in  earlier  work  by  the  PI,  proved  to  be  a  strong  basis  for  further  developments.  This 
class  of  models  has  in  its  core  the  notion  of  gyroscopically  interacting  systems  (i.e.  particles 
subject  to  forces  orthogonal  to  velocities),  in  contrast  with  literature  rooted  in  the  many-body 
problems  of  celestial  mechanics.  We  note  that,  the  study  of  collective  behavior  has  a 
distinguished  role  in  the  foundations  of  physical  science,  as  in  work  on  the  problem  of  stability 
of  the  solar  system  (by  Lagrange,  Laplace  and  their  illustrious  descendants),  and  James  Clerk 
MaxwelFs  investigation  of  the  character  and  stability  of  Saturn’s  rings  (1859).  But  collective 
behavior  as  observed  in  biological  organisms  (birds,  insects,  fish,  dolphins  etc.)  mediated  by 
sensory  modalities  such  as  vision,  bio-sonar  and  olfaction  is  more  naturally  modeled  by  steering 
laws  (i.e.  laws  that  change  direction  of  movement)  under  the  influence  of  such  sensory  feedback, 
in  contrast  to  using  forces  of  attraction  and  repulsion  as  encountered  say  in  celestial  mechanics 
and  chemistry.  This  outlook  led  us  to  the  idea  of  creating  and  studying  collective  behavior  out  of 
dyadic  building-block  interactions  (such  as  motion  camouflage  (MC),  constant  bearing  pursuit 
(CB),  boundary  following  (BF),  etc.). 

As  a  result  of  the  focus  on  gyroscopic  interactions,  a  second  distinctive  feature  of  this  project  is 
that  geometric  ideas  occupy  a  central  place  in  our  conceptual  framework.  Geometry  enters  the 
investigation  of  collective  behavior  from  multiple  vantage  points:  the  structure  of  configuration 
space;  the  synthesis  of  control  strategies;  the  role  of  symmetries  and  reduction  in  closed  loop 
dynamics;  the  analysis  of  empirical  data  from  biology;  and  the  realization  of  control  strategies  in 
autonomous  systems.  In  particular,  the  geometry  of  the  Euclidean  group  in  2  and  3  dimensions 
and  its  role  in  formulating  models  via  theory  of  moving  frames  is  of  central  interest.  Methods 
from  graph  theory  were  used  to  assemble  such  dyadic  interactions  into  collective  behavior  e.g. 
constant  bearing  cyclic  pursuit. 

Third,  we  made  progress  in  the  project  in  these  afore-mentioned  directions,  by  benefiting  from 
close  study  of  low  dimensional  examples.  Building  on  experience  with  data  from  prey-capture 
experiments  involving  echolocating  bats,  we  developed  methods  to  assimilate  sampled 
observations  of  collectives  of  continuous  time  dynamical  systems,  e.g.  predator-prey  encounters 
and  bird  (starlings)  flocking  events,  into  generative  models  with  continuous  time  inputs  and 
outputs.  Results  of  such  assimilation  (reconstruction)  were  used  to  evaluate  hypotheses  of 
interest,  based  on  correlations,  delays,  and  possible  mechanisms  of  interaction  between 
elementary  units  of  the  observed  population.  Using  models  based  on  moving  frames  and  methods 
from  nonlinear  smoothing,  we  found  ways  to  solve  the  problem  of  numerically  reconstructing 


collectives.  We  also  made  effective  use  of  optimal  control  methods  for  computational  schemes, 
and  formulation  of  optimality  principles  for  collective  behavior.  We  further  tested  our  ideas  and 
algorithms  in  a  physical  system  test-bed  based  on  motion  capture  cameras  and  mobile  robots 
acquired  and  installed  in  the  Intelligent  Servosystems  Laboratory,  thanks  to  a  DURIP  award 
through  AFOSR.  Software  tools  developed  in  Maryland  for  data  analysis  have  been  tested  on  a 
variety  of  flocking  events. 

Illustrative  descriptions  of  the  work  done  under  the  project:  Key  outcomes  of  the  project  are 
best  captured  by  the  descriptions  of  two  Ph.D.  dissertations  and  a  Ph.D.  research  proposal  on  a 
dissertation  to  be  defended  in  February  2015. 

(i)  Ph.  D.  Dissertation  of  K.  S.  Galloway  (2011)  -  Cyclic  Pursuit:  Symmetry,  Reduction  and 
Nonlinear  Dynamics.  In  this  dissertation,  we  explore  the  use  of  pursuit  interactions  as  a  building 
block  for  collective  behavior,  primarily  in  the  context  of  constant  bearing  (CB)  cyclic  pursuit. 
Pursuit  phenomena  are  observed  throughout  the  natural  environment  and  also  play  an  important 
role  in  technological  contexts,  such  as  missile-aircraft  encounters  and  interactions  between 
unmanned  vehicles.  While  pursuit  is  typically  regarded  as  adversarial,  we  demonstrate  that 
pursuit  interactions  within  a  cyclic  pursuit  framework  give  rise  to  seemingly  coordinated  group 
maneuvers.  We  model  a  system  of  agents  (e.g.  birds,  vehicles)  as  particles  tracing  out  curves  in 
the  plane,  and  illustrate  reduction  to  the  shape  space  of  relative  positions  and  velocities. 
Introducing  the  CB  pursuit  strategy  and  associated  pursuit  law,  we  consider  the  case  for  which 
agent  pursues  one  other  agent  using  a  CB  pursuit  law.  After  deriving  closed-loop  cyclic  pursuit 
dynamics,  we  demonstrate  asymptotic  convergence  to  an  invariant  submanifold  (corresponding 
to  each  agent  attaining  the  CB  pursuit  strategy),  and  proceed  by  analysis  of  the  reduced  dynamics 
restricted  to  the  submanifold.  For  the  general  setting,  we  derive  existence  conditions  for  relative 
equilibria  (circling  and  rectilinear)  as  well  as  for  system  trajectories  which  preserve  the  shape  of 
the  collective  (up  to  similarity),  which  we  refer  to  as  pure  shape  equilibria.  For  two  illustrative 
low-dimensional  cases,  we  provide  a  more  comprehensive  analysis,  deriving  explicit  trajectory 
solutions  for  the  two-particle  cyclic  case,  and  detailing  the  stability  properties  of  three-particle 
relative  equilibria  and  pure  shape  equilibria.  For  the  three-particle  case,  we  show  that  a  particular 
choice  of  CB  pursuit  parameters  gives  rise  to  remarkable  almost-periodic  trajectories  in  the 
physical  space.  We  also  extend  our  study  to  consider  CB  pursuit  in  three  dimensions,  deriving  a 
feedback  law  for  executing  the  CB  pursuit  strategy,  and  providing  a  detailed  analysis  of  the  two- 
particle  mutual  pursuit  case.  We  complete  the  work  by  considering  evasive  strategies  to  counter 
the  motion  camouflage  (MC)  pursuit  law.  After  demonstrating  that  a  stochastically  steering 
evader  is  unable  to  thwart  the  MC  pursuit  strategy,  we  propose  a  (deterministic)  feedback  law  for 
the  evader  and  demonstrate  the  existence  of  circling  equilibria  for  the  closed-loop  pursuer-evader 
dynamics. 

(ii)  Ph.D.  dissertation  of  M.  Mischiati  (2011)  -  Analysis  and  Synthesis  of  Collective  Motion: 
from  Geometry  to  Dynamics.  The  subject  of  this  dissertation  is  collective  motion,  the 
coordinated  motion  of  two  or  more  individuals,  in  three-dimensional  space.  Inspired  by  the 
problems  of  understanding  collective  motion  in  nature  and  designing  artificial  collectives  that 
can  produce  complex  behaviors  we  introduce  mathematical  methods  for  the  analysis  of  collective 
motion  data,  and  biologically-inspired  algorithms  for  generating  collective  motion  in  engineered 
systems.  We  explore  two  complementary  approaches  to  the  analysis  and  synthesis  of  collective 


motion.  The  first  "top-down"  approach  consists  in  exploiting  the  geometry  of  n-body  systems  to 
identity  certain  elementary  components  of  collective  motion.  A  main  contribution  of  this  thesis  is 
to  reveal  a  new  geometrical  structure  (fiber  bundle)  of  the  translation-reduced  configuration 
space  and  a  corresponding  classification  of  collective  motions  alternative  to  the  classical  one 
based  on  reduction  to  shape  space.  We  derive  a  mathematical  framework  for  decomposing 
arbitrary  collective  motions  into  elementary  components,  which  can  help  identify  the  main 
modes  of  an  observed  collective  phenomenon.  We  synthesize  vector  fields  that  implement  some 
of  the  most  interesting  elementary  collective  motions,  and  suggest,  whenever  feasible, 
decentralized  implementations.  The  second  “bottom-up”  approach  consists  in  starting  from 
known  biologically-plausible  individual  control  laws  and  exploring  how  they  can  be  used  to 
generate  collective  behaviors.  This  approach  is  illustrated  using  the  motion  camouflage 
proportional  guidance  law  as  a  building  block.  We  show  that  rich  and  coordinated  motion 
patterns  can  be  obtained  when  two  individuals  are  engaged  in  mutual  pursuit  with  this  control 
law.  An  extension  of  these  dynamics  yields  coordinated  motion  for  a  collective  of  n  individuals. 

(iii)  Ph.D.  dissertation  proposal  of  B.  Dey  (2013)  -  Reconstruction,  Analysis  and  Synthesis  of 
Collective  Motion.  As  collective  motion  plays  a  crucial  role  in  modern  day  robotics  and 
engineering  it  seems  appealing  to  seek  inspiration  from  nature,  which  abounds  with  examples  of 
collective  motion  (starling  flocks,  fish  schools  etc.).  This  approach  towards  understanding  and 
reverse-engineering  a  particular  aspect  of  the  nature  forms  the  foundation  of  this  research 
proposal,  and  the  main  contribution  of  this  proposal  is  threefold.  First  we  identify  the  importance 
of  appropriate  algorithms  to  extract  parameters  of  motion  from  sampled  observations  of  the 
trajectory,  and  then  by  assuming  a  proper  generative  model  we  turn  this  into  a  regularized 
inversion  problem  with  the  regularization  term  imposing  smoothness  of  the  reconstructed 
trajectory.  First  we  assume  a  linear  triple  integrator  model,  and  by  penalizing  high  values  of  the 
jerk  path  integral  we  reconstruct  the  trajectory  through  an  analytical  approach.  Alternatively,  the 
evolution  of  a  trajectory  can  be  governed  by  natural  Frenet  frame  equations.  Inadequacy  of 
integrability  theory  for  nonlinear  systems  poses  the  utmost  challenge  in  having  an  analytic 
solution,  and  forces  us  to  adopt  a  numerical  optimization  approach.  However  by  noting  the  fact 
that  the  underlying  dynamics  defines  a  left  invariant  vector  field  on  the  Lie  group,  we  develop 
the  initial  framework  (based  on  Pontryagin’s  maximum  principle)  to  obtain  a  semi-analytic 
solution  in  this  case.  Equipped  with  the  appropriate  algorithms  for  trajectory  reconstruction  we 
analyze  flight  data  for  biological  motions,  and  this  marks  the  second  contribution  of  this 
proposal.  By  analyzing  the  trajectory  data  of  bat  flight  in  two  different  settings  (chasing  a  free 
flying  praying  mantis  and  competing  with  a  conspecific  to  catch  a  tethered  mealworm)  we 
provide  evidence  to  show  the  presence  of  a  context  specific  switch  in  flight  strategy.  Moreover, 
our  approach  provides  a  way  to  estimate  the  behavioral  latency  associated  with  these  flights.  On 
the  other  hand,  we  have  also  analyzed  the  trajectory  data  for  flocking  events  by  European 
starlings,  and  it  can  be  concluded  from  our  data  that  the  flock-averaged  coherence  (the  average 
cosine  of  the  angle  between  the  velocities  of  a  focal  bird  and  its  neighborhood  center  of  mass, 
averaged  over  the  entire  flock)  gets  maximized  by  considering  5-7  nearest  neighbors.  Therefore, 
according  to  our  data,  topological  notion  of  distance  plays  a  more  important  role  than  the 
metrical  notion.  The  third  and  final  contribution  of  this  proposal  lies  in  the  domain  of  control  law 
synthesis.  Drawing  inspiration  from  coherent  movement  of  starling  flocks,  we  introduce  a 
strategy  (Topological  Velocity  Alignment)  for  collective  motion,  wherein  each  agent  aligns  its 
velocity  along  the  direction  of  motion  of  its  neighborhood  center  of  mass.  A  feedback  law  has 


also  been  proposed  for  achieving  this  strategy,  and  we  have  analyzed  two  special  cases  (two- 
body  system;  and  an  N-body  system  with  cyclic  interaction)  to  show  effectiveness  of  our 
proposed  feedback  law.  It  has  been  observed  through  numerical  simulation  that  this  approach 
towards  collective  motion  can  give  rise  to  a  splitting  behavior. 

Significance  of  the  work  supported  by  the  AFOSR  grant:  The  work  has  led  to  advancing  our 
understanding  of  collective  behavior  through  fundamental  building-block  interactions  captured 
by  biologically  plausible  feedback  laws.  It  has  also  provided  guidance  on  exploiting  these 
advances  by  implementation  of  such  feedback  laws  in  robot  collectives  in  a  physical  system  test¬ 
bed  at  the  University  of  Maryland.  It  has  advanced  the  technology  of  data  gathering  on  collective 
behavior  and  related  statistical  physics  based  theoretical  analysis,  as  an  outcome  of  the  work  by 
the  collaborators  of  ISC-CNR  in  Rome,  led  by  the  co-PI. 
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Report  on  the  program  of  Experimental  Data  Gathering  and  Theoretical  Analysis  conducted 
under  the  leadership  of  the  Co-PI  Andrea  Cavagna  in  Rome  (sub-awardee  of  University  of 
Maryland) 

Section  1  -  Accomplishments  during  the  first  year  of  performance  (2010-2011)  of  the  group  of  Co- 
PI  Cavagna  in  Rome. 

During  the  first  year  of  performance  (2010-2011)  the  experimental  program  in  Italy  yielded  new 
observations  of  starlings  and  midges  in  natural  settings  with  higher  temporal  resolution  than  achieved 
before.  The  data  was  subject  to  a  series  of  computer  vision  based  analyses  (segmentation,  static 
matching,  dynamic  matching,  static  3D-rconstruction,  cluster  analysis,  stereo  tracking  -  3D 
reconstruction. 

High  Speed  Camera  Verification  Testing 

Three  high  speed  cameras  are  required  for  data  acquisition  of  collective  behavior.  The  camera  selected 
for  this  purpose  is  the  IDT  M5,  which  has  a  maximum  frame  rate  of  170  frames  per  second  (fps)  at  a 
resolution  of  2288x1728.  Verification  tests  were  conducted  on  this  camera  prior  to  purchase.  These 
tests  included:  pixel  sensitivity,  noise  characterization,  image  saving  time,  multiple  camera 
synchronization  and  frame  rate  verification.  The  last  two  tests  were  conducted  using  an  in-house 
custom  designed  target  consisting  of  a  360°  protractor  and  disk  with  needle  connected  to  a  high  speed 
stepper  motor.  The  period  of  revolution  was  calibrated  and  verified  using  an  optical  counter. 

Calibration  Methods 

For  the  earlier  European  union  funded  Starflag  project,  the  camera/lens  combination  was  calibrated  by 
a  third-party  company.  In  the  AFOSR-funded  project  through  the  University  of  Maryland,  all  camera 
calibrations  (intrinsic  and  extrinsic  parameters)  are  carried  out  in-house. 

Intrinsic  Parameters 

Taking  inspiration  from  the  calibration  methods  used  by  the  Bodenschatz  Turbulence  Group  at  the  Max 
Planck  Institute,  the  following  camera  intrinsic  parameters  are  of  interest:  the  focal  length  (Q),  the 
principal  point  (u0,  v0)  and  the  1st  radial  distortion  parameter  (ki).  Limiting  the  number  of  intrinsic 
parameters  reduces  the  errors  produced  during  the  optimization  stage  of  the  calibration  methodology. 

The  core  camera  calibration  function  is  provided  by  the  Open  Computer  Vision  (OpenCV)  library, 
which  is  the  translation  of  the  Camera  Calibration  Toolbox  for  MATLAB  into  C/C++.  The  target  for 
the  calibration  is  a  2D  black  and  white  checkerboard  pattern.  In  a  typical  set  of  calibration  images 
subsequent  images  tend  to  be  related  to  one  another  by  a  rotation,  tilt  or  translation.  It  is  then  possible 
to  introduce  bias  if  a  block  of  consecutive  images  are  used.  Hence  a  new  method  for  determining  the 
intrinsic  parameters  we  developed  to  reduce  target  orientation  bias,  known  as  the  iterative  median 
method.  In  this  method,  a  set  of  M  trials  are  conducted  each  with  an  increasing  number  of  iterations 
( Nm )•  Each  iteration  returns  a  set  of  intrinsic  parameters  based  on  P  randomly  selected  images  from  the 
total  set  of  images  ( T)  of  the  calibration  target  in  different  orientations.  The  first  trial  (M  =  1)  is 
initialized  using  the  theoretical  values  of  the  intrinsic  parameters  -  manufacturer's  focal  length  in 
pixels,  principal  point  at  the  center  of  the  image  and  zero  distortion.  Each  subsequent  trial  (M>  1)  is 
initialized  using  the  median  value  of  the  Nm  intrinsic  parameters  calculated  from  the  previous  trial.  A 
typical  set  of  values  used  in  the  iterative  median  method  are:  M=  5,  Nm=  (10,  20,  50,  100,  1000),  P  = 
20  and  T=  50. 


Extrinsic  Parameters 


The  method  for  determining  the  extrinsic  parameters  (i.e.  the  mutual  Euler  angles  and  translation 
vectors  between  the  3  cameras)  differs  for  the  two  different  collective  groups  that  are  under 
investigation.  For  the  starling  flocks,  the  method  is  identical  to  that  of  the  original  Starflag  project  -  a 
wire  and  various  gauges  are  used  to  align  Cameras  2  and  3  to  known  relative  angles  while  the  position 
and  orientation  of  Camera  1  are  unknown  (only  required  to  aid  in  stereo  matching).  When 
photographing  midge  swarms,  it  is  not  possible  to  wait  in  a  given  location  for  a  swanning  event  to 
enter  the  cameras'  overlapping  ficlds-of-vicw.  Hence  the  alignment  of  the  cameras  using  the  wire 
method  is  not  possible.  We  developed  a  new  method  using  two  targets  separated  by  a  known  distance 
and  zero  temperature  Monte  Carlo  optimization.  This  calibration  method  is  further  described  in  the 
Data  Acquisition  -  Midge  Swarms  section. 


Data  Acquisition  -  Starling  Flocks 

Summary  of  Experimental  Procedure 

Experiments  are  carried  out  on  the  roof  of  Palazzo  Massimo  alle  Tenne  overlooking  the  bus  transit 
center  of  Termini  station.  Cameras  1  and  2  are  located  on  the  right  side  of  the  roof  (when  looking  at 
Termini  station),  approximately  3  m  apart,  while  Camera  3  is  25  m  to  the  left  of  Camera  2.  The 
cameras  were  set  at  80  fps  as  a  compromise  between  temporal  resolution  and  total  length  of  event 
captured  (the  cameras  are  limited  to  a  maximum  of  2500  images  per  acquisition  regardless  of  frame 
rate).  The  three  cameras  are  setup  such  that  the  fields-of-view  overlap  approximately  100  m  from  the 
cameras.  The  cameras  are  set  in  pre-trigger  mode,  meaning  that  the  cameras  continuously  buffer 
images  until  the  trigger  is  depressed  whereby  the  buffer  is  saved  to  hard  disk.  This  means  the  flocking 
behavior  can  be  “previewed”  prior  to  saving,  thus  increasing  the  probability  of  capturing  desirable 
events. 

Experimental  Issues  Resolved 

While  conducting  an  infinity  test  (shooting  targets  at  a  distance  much  larger  than  the  stereo  baseline)  to 
determine  the  angle  correction  factor  (a  parameter  to  correct  for  any  angular  offset  between  the  camera 
sensor  and  the  camera  body)  it  was  determined  that  the  original  mounting  blocks  for  the  IDT  cameras 
were  not  wide  enough  to  prevent  small  rotations  when  mounted  on  the  rigid  camera  bar.  After 
constructing  wider  mounting  blocks,  the  cameras  could  be  mounted  with  more  predictability.  A  second 
issue  resolved  improved  image  sharpness.  Originally,  when  the  lens  (Schneider  28mm)  was  used  at 
low  f#  the  images  were  severely  blurred,  especially  in  the  corners.  After  several  different  focusing 
tests,  it  was  discovered  that  the  lens  mounting  flanges  on  the  cameras  were  not  set  at  the  proper 
distance  from  the  image  sensor.  After  adjusting  the  flange  on  each  camera,  the  image  quality  at  low  f# 
improved  drastically. 

Experimental  Results 

The  starling  season  begins  approximately  at  the  beginning  of  December  and  lasts  till  near  the  end  of 
February.  During  the  past  season  122  starling  flocking  events  were  captured.  94  of  these  were 
captured  prior  to  the  flange  adjustment.  Although  these  acquisitions  are  not  of  the  highest  image 
quality,  they  are  still  usable,  particularly  for  testing  image  processing  algorithms.  Of  the  122,  45  of 
these  events  are  considered  high  quality  in  terms  of  segmentation  potential  and  flock  isolation  (only  a 
single  flock  in  the  field-of-view). 

Data  Acquisition  -  Midge  Swarms 

Summary  of  Experimental  Procedure 


The  experimental  setup  for  capturing  midge  swarms  is  more  compact  than  for  starling  flocks.  Cameras 
1  and  2  are  mounted  on  the  same  camera  bar,  approximately  25  cm  apart.  Camera  3  is  mounted  on  a 
separate  bar.  The  separation  distance  between  Camera  2  and  3  is  governed  by  the  distance  from  the 
cameras  to  the  swarm  based  on  an  estimate  of  the  acceptable  magnitude  of  inter-midge  distance  error. 
A  typical  set  up  has  Camera  2  and  3  about  8  m  from  the  swarm  and  Camera  3  about  4  m  to  the  left  of 
Camera  2  (when  looking  from  behind  Camera  2).  Since  midge  swarm  movements  are  more  erratic  and 
have  higher  velocities,  the  cameras  are  set  at  the  highest  frame  rate  of  170  fps.  The  cameras  are  set  to 
post-trigger  mode  -  images  are  captured  after  the  trigger  button  is  depressed.  The  reason  for  this 
setting  is  described  below. 

Experimental  Issues  Resolved 

In  a  typical  acquisition  session,  two  operators  track  the  swarm  using  the  micrometric  tripod  heads  until 
the  swarm  stabilizes  at  a  fixed  location.  Only  at  that  point  is  a  sequence  of  images  captured.  This 
precludes  the  use  of  the  alignment  wire  as  the  final  position  of  the  cameras  is  a  function  of  the  location 
of  the  swann.  To  determine  the  extrinsic  parameters  of  the  cameras,  a  new  extrinsic  calibration 
technique  was  developed.  The  calibration  rig  (shown  in  Figure  1)  consists  of  two  targets  mounted  on  a 
rigid  bar,  each  consisting  of  two  white  and  two  black  solid  rectangles  in  a  checkerboard  pattern.  This 
pattern  forms  two  perpendicular  lines  that  cross  at  a  point  (target  center).  To  identify  the  two  targets, 
the  targets  have  been  made  such  that  they  are  mirror  images  of  one  another.  The  distance  between  the 
target  centers  (Ar)  can  be  precisely  measured  using  a  laser  to  within  ±0.002  m.  A  set  of  25  images  of 
the  calibration  rig  in  various  orientations  are  taken  in  the  field  once  swarming  activity  has  ended.  The 
target  center  image  locations  are  extracted,  with  sub-pixel  accuracy  of  0.25  pixels,  using  custom 
software  that  requires  the  user  to  manually  click  nearby  the  target  centers.  The  target  center  image 
points  for  two  cameras  are  then  used  to  calculate  the  essential  matrix  of  the  two  cameras  (assumption  is 
that  intrinsic  parameters  of  each  camera  are  available)  using  a  linear  set  of  equations  and  singular  value 
decomposition  (SVD).  The  essential  matrix  can  then  be  decomposed  into  rotation  and  translation 
matrices  (extrinsic  parameters)  that  relate  the  position  of  one  camera  with  respect  to  the  camera  center 
of  the  other.  The  assumption  here  is  that  the  world  reference  frame  is  located  at  the  camera  center  of 
one  camera.  The  final  purpose  of  the  extrinsic  parameters  is  to  reconstruct  the  3D  location  of 
individual  midges  and  of  particular  interest  is  precisely  determining  the  inter-midge  distances.  To 
reduce  the  error  in  the  calculation  of  inter-midge  distances,  the  final  step  of  the  extrinsic  parameter 
calibration  optimizes  the  rotation  and  translation  parameters  by  minimizing  the  distance  error  between 
the  target  center  points.  This  optimization  is  carried  out  using  a  zero  temperature  Monte  Carlo 
algorithm.  Figure  2  illustrates  the  typical  distance  error  between  target  centers  (5(Ar))  for  a  set  of  25 
images  with  a  inter-target  separation  of  Ar  =1  m.  Note  that  the  largest  absolute  error  is  0.002  m  with 
the  average  error  being  approximately  0.0007  m. 


Experimental  Results 

An  initial  set  of  19  image  sequences  have  been  captured  with  14  of  these  sequences  being  of  high 
quality  (reasonable  backgrounds  leading  to  a  higher  probability  of  proper  image  segmentation).  All 
these  sequences  were  captured  at  Villa  Paganini. 

Current  State  of  Experiment 

Data  acquisition  is  in  progress.  The  anticipated  length  of  the  midge  swarming  season  is  from  the 
beginning  of  May  until  the  end  of  October  or  the  middle  of  November  (depending  on  the  weather).  A 
current  experimental  issue  to  be  resolved  is  selecting  a  preferred  location,  which  offers  an  acceptable 
background  (no  regions  of  image  saturation  and  little  or  no  movement),  enough  room  for  experimental 
equipment,  proper  light  conditions  (swarms  tend  to  occur  at  the  interface  between  sunlight  and  shadow) 


and  reasonable  swarm  size  (has  to  fit  within  the  field-of-view  of  a  camera)  and  density  (increasing 
density  reduces  image  segmentation  efficacy).  Villa  Paganini  is  a  strong  candidate  since  the  majority 
of  these  conditions  are  met.  However,  lighting  has  become  an  issue.  A  possible  solution  to  this 
problem  is  to  use  a  landmark  to  move/attract  midge  swarms  to  a  favorable  location.  Experiments  to 
determine  the  correct  landmark  and  the  effectiveness  of  the  landmarks  are  ongoing,  but  initial  tests 
have  shown  that  it  is  possible  to  attract  and  also  move  a  swarm  to  a  more  desirable  location. 

Computer  Vision  Accomplishments 

Due  to  the  acquisition  of  new  high  speed  cameras,  the  sequence  of  digital  stereoscopic  images  of  an 
animal  group  (flock  and/or  swarm)  taken  in  the  new  project  are  much  longer  temporally  with  each 
image  being  of  lower  resolution,  compared  to  the  StarFlag  project.  The  consequence  of  having  higher 
temporal  resolution  is  that  the  computer  vision  analysis  requires  higher  processing  specifications.  Also, 
image  segmentation  of  lower  resolution  images  becomes  more  difficult. 


To  cope  with  these  two  issues,  we  developed  a  new  software  framework,  using  the  paradigm  of  full 
integration  that  can  process  all  images  simultaneously.  This  is  in  contrast  to  the  software  architecture 
employed  in  the  StarFlag  project,  which  analyzed  each  image  separately.  The  new  framework 
increases  the  flexibility  of  processing  algorithms  that  can  be  implemented  as  simultaneous  frame 
analysis  allows  feedback  strategies  to  be  added  along  with  statistical  information  to  be  collected.  The 
latter  can  be  used  to  automatically  set  certain  important  computer  vision  parameters  to  avoid  any 
“hand-tuning”. 

Database:  the  bridge  between  experiment  and  computer  vision 

To  facilitate  the  transfer  of  data  from  experimental  acquisition  to  image  processing,  a  database  is  used 
to  bridge  the  gap.  The  database  not  only  stores  each  biological  event  (starlings  flocking  or  midges 
swarming),  but  also  standardizes  the  parameters  used  to  describe  each  event  and  facilitates  the  retrieval 
of  information  about  each  event.  For  example  event  parameters  of  interest  are:  lenses  used  and 
intrinsic  calibration  parameters  for  lens/camera  pair  and  extrinsic  calibration  parameters.  The  database 
has  been  designed  to  have  various  input  forms  to  reduce  errors  during  data  entry  for  an  event  as  well  as 
an  internet-based  user  interface  to  access  the  data  base. 


Software  architecture 

The  new  software  architecture  can  be  described  from  top-down  with  respect  to  the  processing  steps  for 
a  given  image  sequence: 

•  Segmentation 

•  Static  Matching 

•  Dynamic  Matching 

•  Static  3D-Reconstruction 

•  Cluster  Analysis 

•  Stereo  Tracking:  3D-Trajectories 

The  relationship  between  these  various  software  components  is  shown  in  Figure  3.  Also  shown  in  the 
figure  is  change  status  of  each  the  component  with  respect  to  the  StarFlag  project.  The  remainder  of 
this  section  highlights  some  of  the  key  features  of  each  software  component  listed  above,  with 
emphasis  on  new  implemented  strategies. 


Segmentation 

From  the  images  of  a  biological  event,  the  image  position  of  individual  organisms  can  be  detennined 
using  segmentation  methods.  For  segmentation  to  be  successful  there  must  be  some  way  to  distinguish 
individuals  from  the  background.  Since  the  cameras  used  are  grayscale,  only  contrast  information  can 
be  used.  Starlings  when  imaged  appear  dark  against  the  sky  (a  light  background),  while  for  midges  the 
opposite  occurs  i.e.  midges  are  bright  and  the  background  is  dark.  Once  the  background  has  been 
identified,  it  can  be  removed  from  the  image  by  subtraction  (difference  image).  Next,  a  Gaussian  de¬ 
noting  filter  is  applied  to  reduce  both  image  sensor  noise  and  artifacts  from  the  background 
subtraction.  A  threshold  is  used  to  identify  only  those  pixels  representing  individual  organisms  (binary 
image).  The  new  feature  of  the  segmentation  component  is  the  ability  to  select  the  threshold  parameter 
from  the  properties  of  the  image  histogram  of  the  difference  image.  From  the  binary  image,  pixels  are 
clustered  to  form  “blobs”  that  identify  single  or  multiple  organisms.  Large  blobs  are  split  using  a 
watershed  mechanism.  The  lower  resolution  images  of  the  current  project  contain  more  segmentation 
noise  as  compared  to  those  obtained  in  the  StarFlag  project.  This  disadvantage  means  that  more  care 
has  to  be  taken  in  setting  certain  parameters,  but  can  be  aided  by  examining  global  image  features  such 
as  the  image  histogram.  The  higher  temporal  resolution  of  the  captured  image  sequences  has  lead  to 
implementing  this  component  using  multi-threaded  processes  to  perform  parallel  computing.  So  the 
images  from  each  one  of  the  three  cameras  can  be  processed  simultaneously.  Another  issue 
exaggerated  by  the  higher  temporal  resolution  is  “background  movement”,  which  is  not  easily  removed 
during  background  subtraction.  A  rejection  method  is  implemented  that  removes  any  moving  objects 
that  do  not  belong  to  the  flock  or  swarm  by  clustering  all  segmented  objects  and  following  the  chosen 
animal  cluster  through  all  image  sequences.  An  example  of  the  results  of  segmentation  of  birds  and 
midges  are  shown  in  Figures  4  and  5,  respectively. 


Static  and  Dynamic  Matching 

The  output  of  the  segmentation  process  is  a  set  of  all  segmented  individuals  of  the  analyzed  sequence. 
These  points  then  serve  as  the  input  for  the  static  and  dynamic  matching  algorithms.  These  two 
software  components  provide  solutions  for  stereoscopic  matching  and  tracking,  respectively.  Both 
components  have  as  their  core  algorithm  the  same  pattern  recognition  method  used  in  the  StarFlag 
project,  but  with  added  improvements.  Specifically,  due  to  the  fully  integrated  framework,  the 
parameters  used  in  these  algorithms  can  be  set  automatically  using  feedback  fitting  strategies.  The 
effectiveness  of  the  overall  component  is  seen  by  the  robustness  of  the  final  matches  because  of  the 
increased  number  of  images  available  which  provide  a  solid  statistical  foundation. 


SD-Reconstruction 

From  the  calculated  stereo  matches  between  cameras,  their  3D  position  can  be  reconstructed  using 
standard  algorithms  described  in  computer  vision  literature  (for  example  minimizing  the  residual  of  a 
set  of  equations  linking  the  coordinates  of  the  stereoscopic  image  points  to  the  3D  world  coordinates  of 
an  individual).  The  extrinsic  parameters  from  the  calibration  method  described  in  the  experimental 
summary  are  used  in  this  calculation. 


Cluster  Analysis 

With  the  matching  phase  complete,  connections  between  segmented  individuals  in  different  camera 
views  at  the  same  time  instance  (static  matches)  and  in  different  time  slices  (dynamic  matches)  can  be 
completed.  From  these  connections,  a  graph  can  be  constructed  where  the  nodes  represent  a  segmented 
individual  and  the  links  are  fonned  by  the  static  and  dynamic  matches.  Each  connected  component  of 
this  graph  is  known  as  a  “dynamic  cluster”.  Due  to  the  simultaneous  usage  of  static  and  dynamic 


matches,  links  can  be  formed  between  the  same  individual  in  different  cameras  and  at  different  times 
even  if  the  individual  is  missing  in  certain  frames  because  of  segmentation  problems  (i.e.  the  individual 
was  not  found  in  a  given  frame  due  to  noise  or  being  occluded  by  others). 


The  connectivity  of  the  graph  provides  robustness  to  this  approach  in  that  an  individual  can  be 
recovered  indirectly  simply  by  following  links.  If  all  the  matches  in  a  dynamic  cluster  are  correct,  then 
it  represents  a  real  trajectory  (full  or  partial)  of  an  individual  organism  as  shown  in  Figures  6  (birds) 
and  7  (midges).  Note  in  Figure  6  the  modulations  in  the  trajectories,  which  are  due  to  the  flapping 
wings  of  the  birds.  This  feature  is  only  discoverable  with  the  higher  temporal  resolution  of  the 
cameras.  Due  to  mismatching  errors  the  dynamic  clusters  typically  contain  entangled  trajectories  i.e. 
links  form  between  two  different  dynamic  clusters  (see  Figures  8  and  9).  To  measure  the  properties  of 
the  dynamic  clusters,  three  parameters  are  defined:  the  number  of  frames  in  an  image  sequence  is  T,  the 
average  length  of  all  dynamic  clusters  is  L,  and  the  average  number  of  trajectories  a  given  dynamic 
cluster  are  entangled  with  for  its  entire  length  is  known  as  the  thickness  W  (for  a  real  trajectory  W 
should  be  1).  Currently,  analysis  is  ongoing  on  the  best  strategies  to  disentangle  dynamic  clusters  and 
build  longer  dynamic  clusters.  The  goal  then  is  to  have  L  as  large  as  possible  (ideally  a  ratio  between  L 
and  T  should  be  one)  and  W  near  to  one  (or  at  least  less  than  a  fixed  threshold  value). 


A  possible  strategy  to  build  longer  dynamic  clusters  is  to  only  use  a  subset  of  all  the  static  and  dynamic 
matches  available,  based  on  a  given  criteria  such  as  coherence  on  stereoscopic  and/or  dynamic 
matches.  For  example,  using  only  static  links  that  have  complete  coherence  between  all  three  cameras 
(i.e.  the  same  individual  can  be  located  and  linked  in  each  camera  at  the  same  time  instance)  or  using 
all  dynamic  links  with  complete  coherence  (i.e.  the  same  individual  can  be  tracked  across  multiple 
frames  in  a  single  camera)  over  a  set  number  of  frames.  To  determine  the  most  effective  strategies,  the 
average  value  of  W  for  all  clusters  versus  L  for  various  strategies  are  plotted  as  shown  in  Figure  10  for 
a  single  image  acquisition. 


Stereo  Tracking:  3D-Trajectories 

Once  a  “best  strategy”  is  selected  the  dynamic  clusters  can  be  disentangled  and  trajectories  formed  by 
merging  as  many  pieces  as  possible.  This  work  is  ongoing  with  the  current  solution  to  use  a  Monte 
Carlo  algorithm  with  a  cost  function  based  on  reducing  the  number  of  inter-cluster  links.  The 
disentanglement  forms  2D  real  trajectories,  but  these  can  be  translated  in  to  3D  trajectories  using  the 
stereoscopic  matches  generated  by  the  3D  Reconstruction  component. 


Figures 


Figure  1  (20110511_cam3_000002.jpg):  Image  for  midge  swarm  extrinsic  parameter  calibration 
showing  the  new  calibration  target. 
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Figure  2  (delta(Deltar).png):  Typical  absolute  distance  error  between  target  centers  of  the  extrinsic 
parameter  calibration  rig  in  real  world  coordinates.  Distances  are  calculated  from  the  optimal  rotation 
and  translation  matrices  returned  by  the  zero  temperature  Monte  Carlo  algorithm. 


Figure  3  (SW_Architccturc.pdf)  :  Software  Architecture  Hierarchy  (The  yellow  balloons  show  if  the 
software  components  are  new  or  have  been  improved  in  the  new  project  compared  to  the  Starflag  one). 


Figure  4  (fde:  FLOCK.jpg):  Typical  flock  of  starlings,  with  segmented  individuals  numbered  and 
highlighted  with  a  blue  contour. 


Figure  5  (file:  SWARM.jpg):  Typical  swarm  of  midges,  with  segmented  individuals  numbered  and 
highlighted  with  a  blue  contour. 
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Figure  6  (file:  flock_cluster_l.jpg):  A  typical  dynamic  cluster  of  a  starling  acquisition;  the  three  lines 
together  represent  a  single  real  trajectory  as  seen  by  each  of  the  three  cameras.  Each  point  of  the  line 
represents  the  baricenter  coordinates  of  the  bird  obtained  by  the  segmentation  software.  Note  the 
modulation  of  the  trajectory  which  is  due  to  flapping  wings  of  the  bird,  which  is  only  discoverable  with 
a  high  temporal  resolution. 
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Figure  7  (file:  swarm_cluster_l.jpg):  Atypical  dynamic  cluster  of  a  midge  acquisition;  the  three  lines 
together  represent  a  single  real  trajectory  as  seen  by  each  of  the  three  cameras.  Each  point  of  the  line 
represents  the  baricenter  coordinates  of  the  midge  obtained  by  the  segmentation  software. 


Flock:  dynamic  cluster  2 


Figure  8  (file:  flock_cluster_2.jpg):  A  typical  dynamic  cluster  of  a  starling  acquisition;  here  there  are 
entangled  trajectories  as  shown  by  the  multi-colored  data  points  in  each  line.  Birds  belonging  to 
different  trajectories  have  the  same  color  at  the  same  frame  for  each  camera.  Each  point  of  the  line 
represents  the  baricenter  coordinates  of  the  bird  obtained  by  the  segmentation  software.  Note  the 
modulation  of  the  trajectory  which  is  due  to  flapping  wings  of  the  bird,  which  is  only  discoverable  with 
a  high  temporal  resolution. 


Swarm:  dynamic  cluster  2 


Figure  9  (file:  swarm_cluster_2.jpg):  A  dynamic  cluster  of  a  midge  session;  here  there  are  entangled 
trajectories  as  shown  by  the  multi-colored  data  points  in  each  line.  Midges  belonging  to  different 
trajectories  have  the  same  color  at  the  same  frame  for  each  camera.  Each  point  of  the  line  represents 
the  baricenter  coordinates  of  the  midge  obtained  by  the  segmentation  software. 
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Figure  10  (file:  Strategy_W_vs_L.jpg):  Average  cluster  thickness  versus  average  cluster  length  for 
various  cluster-building  strategies  (represented  by  different  colored  and  shaped  symbols).  The  black 
arrow  indicates  the  “best”  strategy  for  this  session  that  gives  long  (L/T  ~  0.80)  and  thick  (W  ~  5.5) 
clusters,  which  consists  of  using  75%  of  static  links  that  are  completely  coherent  and  all  dynamic  links 
that  are  completely  coherent  up  to  two  frame  levels. 


Section  2  -  Accomplishments  during  the  second  year  of  performance  (2011-1012)  the  group 
of  Co-PI  Cavagna  in  Rome: 


•  Finished  experimental  data-taking  campaign,  both  for  birds  and  for  midges. 

•  Developed  a  novel  tracking  method  (with  respect  to  the  last  reporting  period)  that  led  to 
solving  a  range  of  remaining  computer  vision  problems  and  reconstruction  of  all 
individual  trajectories  in  several  flocks  of  starlings  and  midge  swarms. 

•  The  tracking  algorithm  was  run  on  several  flocking  events,  and  on  some  swarming 
events. 

•  At  the  end  of  the  second  year  3D  trajectories  of  starling  flocks  were  delivered  to  the  PI 
Krishnaprasad  and  his  students  for  smoothing  and  extraction  of  curvature  controls. 

•  Analyzed  the  data  and  developed  new  theoretical  approaches  and  numerical  models  of 
collective  animal  behavior. 

•  Wrote,  submitted  and  published  results  in  high  profde  peered-reviewed  journals. 


Deliverables 


At  the  end  of  the  reporting  period  (more  precisely  on  May  30th,  2012),  Co-PI  Cavagna,  head  of 
the  ISC-CNR  unit,  traveled  to  the  US,  Maryland  and  discussed  with  the  PI  Krishnaprasad 
(UMD)  the  state  of  the  project.  In  that  occasion,  Cavagna  delivered  to  the  PI  a  set  of 
reconstructed  3d  trajectories  of  flocking  birds.  These  are  the  delivered  events,  with  their 
specifications: 

•  20111209  -  20  birds  -  8.8  secs  @  170  fps 

•  20111125  -  50  birds  -  5.9  secs  @  170  fps 

•  20111222  -  60  birds  -  3.6  secs  @  170  fps 

•  20110208  -  179  birds  -  5.5  secs  @  80  fps 


Publications  Status  at  this  stage  of  the  project  was  as  follows 

1.  Statistical  mechanics  of  natural  flocks  of  birds  W.  Bialek,  A.  Cavagna,  I.  Giardina,  T. 
Mora,  E.  Silvestri,  M.  Viale,  A.  Walczak,  Proc.  Natl.  Acad.  Sci.  US  109,  4786  (2012) 

2.  Starling  flock  networks  manage  uncertainty  in  consensus  at  low  cost  G  F.  Young,  L. 
Scardovi,  A.  Cavagna,  I.  Giardina,  N.  E.  Leonard  (first  Submitted  in  March  2012  - 
Published  in  Feb  2013  in  PLOS  Computational  Biology  9  (1),  el002894) 

3.  Spatially  balanced  topological  interaction  grants  optimal  cohesion  in  flocking  models  M. 
Camperi,  A.  Cavagna,  I.  Giardina,  G.  Parisi,  E.  Silvestri,  Interface  Focus  10.1098/ 
rsfs.20 12.0026  (2012) 


4.  Diffusion  in  starling  flocks  A.  Cavagna,  S.  M.  Duarte  Queiros,  I.  Giardina,  F.  Stefanini 
and  M.  Viale  ArXiv:  1206443  (published  in  Feb  2013  in  Proc.  Royal.  Soc.  B  280,  1756) 


5.  Boundary  information  inflow  enhances  correlation  in  flocking  A.  Cavagna,  I.  Giardina,  F. 
Ginelli  ArXiv:  0502936  (accepted  in  Physical  Review  Letters  -  to  be  published  2013) 


Data  Gathering  Experiments  Conducted  by  Co-PI’s  group 


Introduction 

From  June  2011  to  June  2012  experiments  on  starling  flocks  and  midge  swarms  were  carried  out 
on  a  daily  basis.  Equipment,  experimental  set-up  and  calibration  software  of  intrinsic  and 
extrinsic  camera  parameters  were  the  same  as  the  previous  year.  Flocking  and  swanning  events 
were  captured  using  a  system  of  three  synchronized  high  speed  cameras  IDT  M5.  Two  of  these 
cameras,  the  stereometric  pair,  are  placed  25m  apart  for  birds  and  about  3m  apart  for  midges.  A 
third  camera  sits  at  a  shorter  distance  from  a  stereometric  one  (2.5m  for  birds  and  0.25m  for 
midges).  Set-up  for  midges  included  also  an  omnidirectional  hot-wire  anemometer  synchronized 
with  the  camera  system.  It  is  positioned  as  close  as  possible  to  the  swarm,  so  that  a  measure  of 
the  wind  speed  nearby  the  swarm  can  be  retrieved  and  gusts  of  wind  can  be  detected.  Fig.  1 
shows  a  typical  set-up  in  the  field  for  midges  experiment. 


Starling  Flocks 

New  data  on  starling  flocks  were  acquired  from  December  2011  to  February  2012.  The  main 
location  for  the  experiment  was  Piazza  dei  Cinquecento  on  the  roof  of  Palazzo  Massimo. 
Unfortunately,  due  to  unknown  biological  reasons,  birds  moved  from  the  roosting  site  in  Piazza 
dei  Cinquecento  on  January.  This  forced  the  experiment  to  be  moved  to  another  roosting 
location,  K  Building  of  Poste  Italiane  Headquarters  in  Eur,  where  the  experimental  set-up  was 
limited  to  a  non-ideal  area.  Hence,  few  events  were  captured  at  this  location. 

The  experiment  on  flocks  was  focused  in  particular  on  two  different  goals:  big  flocks  (up  to 
thousand  birds),  and  flocks  which  are  turning.  To  achieve  the  first  goal,  new  lenses  were  needed. 
In  fact,  due  to  their  narrow  field  of  view,  Schneider  28mm  were  not  suitable  for  this  purpose.  In 
particular  Zeiss  Distagon  T*  25mm,  Zeiss  Distagon  T*  21mm,  Kowa  LM16XC  16mm  were 
tested  and  used  on  the  field.  During  turns,  the  probability  of  optical  occlusions  is  higher  than  in  a 
flock  passing  straight  in  the  field  of  view.  In  order  to  have  more  details  during  turns,  a  frame  rate 
of  170  fps  was  chosen,  instead  of  80  fps  as  in  the  previous  year. 

The  total  number  of  acquisitions  is  65  and  among  them  53  are  of  high  quality.  Histogram  in  Fig. 
2  shows  number  of  usable  and  non-usable  acquisitions  comparing  season  2010-2011  with  the 
current  one.  From  this  plot,  it  is  clear  that  in  this  last  year  the  percentage  of  high  quality 
acquisitions  increased  considerably,  even  if  the  total  number  of  recorded  events  is  much  smaller 
than  the  previous  year. 


Midge  Swarms 


Swarming  events  were  recorded  in  Villa  Paganini,  Villa  Ada,  Parco  degli  Acquedotti,  Laghetto 
dell’Eur.  Midge  pictures  are  taken  using  back-scattering  sunlight,  and  a  dark,  uniform,  stationary 
background  is  needed  to  provide  high  contrast.  Taking  pictures  using  back-scattering  implies 
having  cameras  almost  in  front  of  the  sun,  with  a  consequent  glare  in  the  pictures.  Glare 
dramatically  reduces  image  quality.  In  fact,  whenever  a  midge  passes  in  front  of  a  glared  region 
it  disappears  and  it  is  not  possible  to  retrieve  its  trajectory.  Black  PVC  lens  hood,  0.45m  long, 
were  built  in  home  and  used  to  reduce  glare  effect  on  the  pictures. 

A  black  fabric  background,  3x1 5m  in  size  is  mounted  behind  the  swann,  see  Fig.  1,  to  improve 
contrast,  to  cover  bright  objects  (i.e.  grass,  leaves)  and  to  avoid  moving  faraway  objects  (i.e. 
people,  cars)  in  the  pictures.  Picture  quality  was  also  improved  using  IR  fdters,  to  reduce  IR- 
radiation  passing  through  the  background  fabric. 

Artificial  background  and  IR  filters  are  not  always  sufficient  to  record  a  usable  acquisition. 
Swarms  often  occur  at  the  interface  between  sunlight  and  shadow.  This  can  create  problems  in 
the  identification  of  midges  when  in  the  shade,  due  to  the  low  contrast.  A  very  useful  tool  is  a 
landmark,  with  which  the  swarm  could  be  moved  to  a  suitable  position.  Several  landmarks  were 
tested,  such  as  plywood  squares  with  black  and  white  features  or  black  and  white  cardboard.  The 
landmark  that  works  better,  in  particular  with  Chironomidae,  is  a  reflective  aluminum/plastic 
wind-shield  sunscreen.  Using  this  landmark,  an  already  formed  swarm  can  be  easily  moved  to  a 
desired  location.  Moreover,  this  landmark  is  a  very  effective  midge  attractor.  So,  it  can  be  used  to 
induce  the  fonnation  of  a  swarm  in  the  best  position  for  the  experiment. 

The  total  number  of  acquired  swarm  events  is  90.  Histogram  in  Fig.  3  shows  distributions  of 
acquisitions  grouped  by  overall  mark  and  size  of  the  swann.  As  this  histogram  shows,  most  of 
the  recorded  events  are  of  small  size  swarms,  up  to  50  midges.  In  the  future  midge  experiment 
will  be  focused  on  capturing  large  swann  events,  with  more  than  200  individuals. 


Midge  identification 

For  each  session  in  the  field,  a  few  midges  from  the  swarm  of  interest  are  caught  with  a  net  and 
preserved  in  75%  alcohol  in  PRC  tubes.  These  samples  are  then  analyzed  in  the  laboratory  to 
identify  the  midge.  Identification  is  a  complex  and  time-consuming  process.  Midges  studied  by 
our  group  are  divided  into  two  families  Ceratopogonidae  and  Chironomidae,  and  their  family  is 
easily  identified  looking  at  antennas.  Once  the  family  is  known,  many  details  of  the  midge 
components  (wings,  eyes,  legs)  have  to  be  observed  to  find  its  genus.  In  particular,  looking  at  the 
hypopigium  (male  sexual  organ),  it  is  possible  to  uniquely  identify  a  midge.  Two  microscopes  are 
used  for  this  purpose.  Some  characteristics  are  observed  through  a  stereometric  microscope,  but 
minute  details  can  only  be  seen  with  an  optical  microscope,  with  the  midge  mounted  on  a  glass 
slide. 


Samples  caught  during  the  reporting  period  (2011-2012)  season  belong  to  both  families, 
Ceratopogonidae  and  Chironomidae.  Most  of  them  are  Chironomidae  and  among  them  only  two 
subfamilies  were  found  till  now,  Orthocladiinae  and  Chironominae.  Going  further  down  in  the 
taxonomy,  several  tribes  were  discovered.  Among  Ceratopogonidae,  both  the  subfamilies 
Dashyelea  and  Forcipomyia  were  found.  In  Fig.4,  a  male  and  a  female  of  Isocladius  sylvestris, 
family  Chironomidae,  subfamily  Orthocladiinae  tribe  Cricotopus,  one  of  the  most  frequent 
caught  midges  are  shown. 


Tracking 


Introduction 

The  Artswann  Computer  Vision  team  developed  a  novel  tracking  algorithm  in  the  period  2011, 
June  1st  -  2012,  May  30th.  The  development  of  the  new  algorithm  has  been  deemed  necessary  in 
order  to  overcome  some  of  the  limitations  of  the  previous  algorithm,  improve  the  tracking  results 
of  flocking  birds  obtained  during  the  previous  year,  and  solve  the  problems  inherent  to  a  different 
kind  of  experimental  data:  swarming  midges. 

The  architecture  of  the  previous  tracking  software  was  based  on  six  processing  steps: 
segmentation;  static  matching;  dynamic  matching;  cluster  analysis;  stereo  tracking;  static  3D- 
reconstruction.  The  current  software  uses  exactly  the  same  modules  to  perfonn  the  segmentation 
and  the  static  3D-reconstrucion  steps,  while  the  two  matching  modules  and  the  tracking  module 
have  been  completely  redesigned. 

The  quality  of  the  3D-trajectories  of  flocking  birds  recovered  with  the  new  software  improved 
significantly  compared  to  the  results  of  the  previous  year.  The  improvement  is  seen  in  terms  of 
trajectory  length  (reduced  trajectory  fragmentation),  total  number  of  tracked  objects  (larger 
flocks,  above  1000  individuals),  and  acquisition  duration  (events  longer  than  1000  frames). 

Limitations  of  the  old  software 

The  software  developed  in  the  first  reporting  period  was  not  able  to  cope  with  long-lasting 
optical  occlusions,  due  to  a  pure  one-to-one  linking  which  forces  every  object  detected  in  the 
images  to  belong  to  only  one  trajectory.  In  Fig.  5,  we  show  a  scheme  of  a  partial  temporal 
sequence  of  two  objects  A  and  B  as  seen  by  one  camera.  The  two  objects  overlap  in  the  image  for 
one  or  more  frames,  three  in  the  example.  The  old  algorithm  would  assign  only  one  dynamic  link 
to  each  detected  object,  therefore  the  points  of  occlusion  would  belong  to  only  one  uninterrupted 
trajectory,  while  the  second  recovered  trajectory  would  result  as  a  broken  one.  This  produced  a 
severe  trajectory  fragmentation.  The  study  of  large  groups  of  animals  and,  in  particular,  the 
analysis  of  the  interactions  among  individuals,  can  be  strongly  biased  by  fragmented  and  missing 
trajectories  in  the  reconstructed  data.  Therefore  it  is  of  utmost  importance  to  track  all  individuals, 
without  losing  their  identities  within  the  duration  of  the  entire  acquisition,  even  when  their 
optical  density  in  the  images  becomes  important. 


A  novel  tracking  approach 


The  new  algorithm  is  based  on  three  main  ideas: 


1 .  The  bifurcation  of  dynamical  links  and  the  construction  of  all  possible  paths  (Fig  7).  In 
the  example  in  Fig. 6  the  assignment  of  multiple  links  results  in  four  uninterrupted  paths, 
of  which  two  are  real  trajectories  and  two  are  hybrid.  Therefore  we  need  a  criterion  to 
select  the  two  right  trajectories. 

2.  A  global  optimization  criterion  to  select  the  correct  paths  out  of  all  possible  paths. 

3.  A  recursive  algorithm,  which  breaks  the  entire  temporal  sequence  into  shorter  intervals, 
and  iteratively  combines  the  partial  solutions  into  the  full  one. 

Static  and  dynamic  linking 

The  strategy  to  find  both  static  (stereo)  and  dynamical  links  is  essentially  the  same  as  in  the  old 
algorithm,  but  with  one  crucial  difference.  Previously,  given  an  object  and  a  set  of  potentially 
good  links  connecting  this  object  to  another  one,  we  had  to  pick  up  one,  using  some  quality 
criterion.  Now,  we  take  all  of  them,  both  at  the  stereo  and  at  the  dynamical  level.  Once  this  is 
done,  we  percolate  all  the  dynamical  links  across  the  whole  acquisition,  and  we  build  all  possible 
paths  (Fig. 7).  The  correct  trajectory  is  certainly  within  this  pool  of  paths,  we  simply  have  to 
select  it.  This  is  done  by  the  global  optimization. 

Global  optimization 

In  Fig.  8,  we  explain  graphically  how  the  optimization  selects  the  two  correct  traces  over  the  four 
candidates  proposed  by  the  multiple  link  assignment  in  Fig.  6.  In  the  example,  the  score  function 
is  defined  as  the  sum  of  the  static  links  between  the  points  of  each  2D-trace  in  different  views. 
The  traces  of  the  two  real  objects  A  and  B  partly  overlap  in  the  left  camera  view,  generating  four 
possible  paths.  On  the  other  hand  in  the  right  camera  there  is  no  overlap,  so  there  are  only  2 
paths.  To  each  pair  of  left-right  paths  is  assigned  a  score  proportional  to  the  number  of  stereo  link 
(same  color)  connecting  the  pair.  Then,  a  global  optimization  is  performed,  finding  the 
assignment  that  maximizes  the  score.  Optimization  is  performed  using  the  C++  APIs  of  IBM 
ILOG  CPLEX  Optimization  Studio  v!2.2. 


Recursive  strategy 

The  number  of  possible  paths  obtained  percolating  the  dynamic  links  grows  exponentially  with 
the  time  duration  of  the  acquisition  (typically  hundreds  or  thousands  of  frames).  Indeed,  the 
number  of  possible  paths  N*  can  be  estimated  as  N*=NeA(bT),  where  N  is  the  number  of  objects, 
T  the  duration  of  the  acquisition,  and  b  the  bifurcation  coefficient.  Fig.  9  reports  the  actual 
number  of  possible  paths  for  a  typical  acquisition  of  roughly  179  birds  as  a  function  of  the 
considered  duration  T  (each  dot  refers  to  one  interval  of  T  frames). 

To  reduce  this  exponential  complexity,  we  implemented  a  recursive  algorithm.  The  acquisition  is 
divided  into  temporal  intervals  with  length  T_1<T,  over  which  the  complexity  of  the  problem  can 
be  handled.  The  global  assignment  is  performed  over  each  interval.  Considering  the  resulting 
matched  2D-traces  of  objects  as  meta-objects,  the  procedure  is  iterated  to  match  the  2D-traces  of 


meta-objects  across  the  different  views  and  over  time  intervals  of  length  T_2  meta-objects.  As 
shown  in  Fig.  10,  the  procedure  is  applied  recursively,  until  the  product  of  the  duration  of  the 
intervals  of  each  iteration  equals  the  duration  of  the  entire  acquisition,  T_1  T_2  ...T_k  =T,  so  that 
the  partial  solutions  retrieved  at  each  iteration  are  combined  into  the  solution  of  the  full  problem 
at  the  last  k-th  iteration. 

Results 

The  implemented  algorithm  has  been  tested  in  processing  1 1  field  acquisitions  of  flocking  birds 
(see  Table  1),  and  2  field  acquisitions  of  swarming  insects. 

We  show  the  reconstructed  trajectories  for  one  of  the  events,  the  bird  flock  labeled 
20111 125_a2_e07  in  Table  1 .  In  Fig.  1 1 ,  the  2D-traces  reconstructed  in  the  image-space  of  each 
of  the  three  cameras  used  to  acquire  the  event  are  shown  for  a  single  bird.  Fig.  12  shows  the 
reconstructed  3D-trajectory  of  the  same  bird.  In  Fig.  14,  we  report  the  nonnalized  lengths 
distribution  of  the  reconstructed  trajectories  for  five  bird  flock  events.  As  shown,  the 
distributions  are  strongly  peaked  towards  1,  indicating  that  the  majority  of  the  reconstructed 
trajectories  are  full-length  ones  and  the  trajectory  fragmentation  is  negligible. 

Fig.  15  shows  the  reconstructed  trajectories  for  an  analyzed  event  of  swanning  midges  (roughly 
100  midges,  2200  frames).  The  new  algorithm  improved  considerably  the  quality  of  the  results, 
and  further  development  of  the  algorithm  and  optimal  tuning  of  its  parameters  to  process  midge 
swarm  data  is  ongoing. 


Data  analysis  and  theoretical  description 


Maximum  Entropy  Approach 

One  of  the  main  objectives  of  our  work  is  to  use  experimental  data  to  infer  information  on  the 
mechanisms  regulating  collective  behavior,  in  particular  the  interactions  at  play  between 
individuals.  The  first  step  in  this  program  is  to  measure  relevant  statistical  observables,  and 
eventually  use  them  as  a  starting  point  for  inference  methods.  In  a  previous  work,  we  measured 
correlations  between  individual  velocities  in  starling  flocks,  and  found  that  they  are  scale  free 
and  unusually  strong,  both  for  what  concerns  correlations  between  flight  directions  and 
correlation  between  speeds.  This  means  that  a  behavioral  change  in  direction  or  speed  of  one  bird 
in  the  flock  influences  all  other  individuals  in  the  group.  Clearly  this  feature  is  crucial  to  grant 
the  group  efficiency  in  response  and  coherence,  but  its  microscopic  origins  were  not  understood. 

In  the  reporting  period  we  developed  new  theoretical  techniques  for  statistical  inference 
based  on  the  Maximum  Entropy  approach,  and  applied  this  method  to  starling  flocks  using 
experimental  correlations  of  flight  directions  as  an  input.  In  this  way,  we  showed  that  the 
interactions  between  individuals  giving  rise  to  the  experimentally  observed  correlations  are  local, 
pairwise,  and  depending  on  the  topological  distance.  From  the  data  we  were  also  able  to  the 
strength  and  the  range  of  the  interactions.  The  model  we  built  using  the  maximum  entropy 
method  finally  fits  perfectly  well  the  observed  correlation  functions,  with  no  free  fitting 


parameters.  Publication  #1:  Bialek  at  al. 


Robustness  of  consensus  in  flocks 

We  addressed  the  issue  of  stability  and  robustness  of  the  cohesion  displayed  by  starling  flocks.  A 
few  years  ago,  we  discovered  that  interactions  between  individuals  in  a  flock  are  topological, 
each  bird  interacting  approximately  with  the  seven  proximate  neighbors  irrespective  of  mutual 
distances.  We  suggested  this  kind  of  interactions  could  be  functional  to  ensure  robust  cohesion  to 
the  flock.  During  the  reporting  period  we  worked  to  consolidate  this  conclusion  by  using  control 
theory  methods.  Using  experimental  data  on  individual  positions,  we  analyzed  robustness  to 
uncertainty  in  multiple  flocks  and  showed  that  interaction  networks  with  7  neighbors  optimize 
the  trade-off  between  group  cohesion  and  individual  effort  to  keep  track  of  the  neighbors.  This 
result  is  quite  remarkable,  as  it  has  been  discovered  on  starling  flocks,  but  it  can  be  applied  to 
any  other  kind  of  network,  both  natural  and  artificial,  showing  the  immense  interdisciplinary 
potential  of  studying  collective  animal  behavior.  Publication  #2:  Young  et  al. 


A  new  spatially  balanced  model 

Numerical  models  are  a  powerful  tool  to  investigate  hypothesis  suggested  by  experiments,  and 
fully  understand  the  mechanisms  leading  to  the  behavior  that  we  observe.  As  mentioned  above, 
we  found  that  interactions  in  flocking  birds  have  a  topological  rather  than  a  metric  nature,  and 
speculated  that  this  kind  of  interactions  is  able  to  grant  the  group  a  more  robust  cohesion.  To  test 
this  conclusion,  we  considered  3D  flocking  models  with  different  interaction  rules,  either  metric 
or  topological,  and  asked  what  kind  of  rules  performed  more  efficiently  in  terms  of  cohesion.  By 
comparing  the  response  of  the  system  to  obstacles  and  noise,  we  concluded  that  spatially 
balanced  topological  interactions  grant  optimal  cohesion  to  the  group.  This  means  that  the 
individuals  each  bird  chooses  to  interact  with  must  be  evenly  distributed  around  it.  This  kind  of 
interaction  grants  a  much  higher  cohesion  of  the  network.  Publication  #3:  Camperi  et  al. 


Diffusion 

All  the  statistical  analysis  described  so  far,  exploits  data  on  individual  3D  positions  and 
velocities.  Availability  of  3D  trajectories,  made  possible  by  the  AFOSR  funding,  opens  new 
opportunities  of  investigation.  One  important  aspect  of  collective  motion  is  related  to  the  fact 
that  individuals  continuously  move  and  exchange  positions.  The  dynamical  nature  of  the 
interaction  network  and  its  role  in  stabilizing  global  ordering  have  been  discussed  theoretically, 
but  never  measured  in  real  flocks.  We  have  measured  the  diffusion  properties  in  starling  flocks 
and  quantified  the  dynamics  of  the  adjacency  network  between  individuals,  discussing 
consistency  with  theoretical  and  numerical  predictions.  We  find  that  birds  have  a  superdiffusive 
motion  (i.e.  faster  than  a  random  walk)  in  the  reference  frame  of  the  flock,  and  that  this  ensures 
that  neighbors'  reshuffling  does  happen.  Therefore,  the  interaction  network  is  not  static. 
However,  we  also  find  that  the  time  scale  over  which  such  reshuffling  takes  place  is  quite  slow, 
when  compared  to  the  individual  relaxation  time.  We  concluded  that  the  enhancement  of  the 
interaction  due  to  this  dynamical  effect  is  probably  smaller  than  expected  in  bird  flocks. 


Publication  #4:  Cavagna  et  al. 


Anomalous  long-range  correlations 

Another  interesting  problem  concerns  the  behavior  of  velocity  correlations  in  starling  flocks. 
These  correlations  are  unusually  strong,  a  feature  that  cannot  be  explained  within  existing 
theories  and  models.  We  put  forward  the  hypothesis  that  the  slow  decay  of  the  correlation  might 
be  due  to  an  infonnation  transfer  from  the  boundary  to  the  bulk  of  the  flock.  To  investigate  this 
issue,  we  performed  numerical  simulations  on  a  simple  alignment  model  subject  to  a  dynamical 
perturbation  on  the  boundary.  We  showed  that,  under  appropriate  conditions,  the  effect  of  the 
perturbation  is  to  trigger  anomalously  long-range  correlations,  as  the  ones  experimentally 
observed.  This  result  seems  to  imply  that  flocks  are  always  close  to  some  state  of  dynamical 
excitation,  where  a  continuous  change  of  state,  and  its  consequence  transfer  of  information  from 
one  side  to  another  of  the  flock,  achieves  the  crucial  result  to  keep  the  whole  flock  highly 
correlated  in  space.  Publication  #5:  Cavagna  et  al. 

Collective  decision-making 

At  the  end  of  the  reporting  period  we  started  addressing  one  of  the  most  intriguing  aspects  of 
self-organized  collective  behavior,  namely  the  process  of  group  decision-making.  What  is  that 
triggers  a  collective  decision?  How  is  the  decision  transferred  to  all  individuals?  These  questions 
are  difficult  to  answer  experimentally,  as  they  require  a  complete  reconstruction  of  individual 
trajectories  in  large  groups  and  for  long  enough  timescales.  Luckily,  our  novel  3D  tracking 
algorithm  (see  Section  :  Tracking)  allowed  us  to  achieve  such  result.  Using  the  newly  retrieved 
3D  trajectories  for  flocking  birds,  we  could  quantitatively  characterize  the  formation  (from 
individual  actions)  and  the  spreading  of  a  collective  decision.  More  precisely,  we  are  focusing  on 
collective  turns  of  flocks.  Our  preliminary  results  are  very  exciting,  as  we  seem  to  find  a  new  set 
of  equations,  describing  collective  turns,  which  are  very  reminiscent  of  superfluid  theory  in 
condensed  matter.  We  are  at  present  consolidating  our  results  by  analyzing  more  events,  and 
expect  to  submit  a  paper  to  a  high-impact  journal  in  a  few  months. 

Ongoing  work. 

Maximum  entropy  on  speed  correlations 

We  are  generalizing  the  maximum  entropy  approach  to  address  not  only  flight  directions  but  also 
individual  speeds.  Interestingly,  preliminary  results  indicate  that  flocks  are  very  close  to  a  critical 
point.  In  other  terms,  the  scale-free  correlations  between  speeds  that  we  observed  experimentally 
are  a  quasi-critical  effect.  A  very  debated  and  open  question  is  whether  biological  systems  are 
poised  to  criticality,  and  whether  there  are  deep  theoretical  and  adaptive  principles  behind  this. 
Our  results  are  likely  to  give  a  strong  contribution  to  this  issue.  At  the  same  time,  we  are 
developing  a  numerical  model  with  variable  individual  speeds  directly  inspired  by  the  maximum 
entropy  approach  results. 
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Fig.  1  -  Typical  set-up  for  midge  experiment. 


Fig.  2  -  Comparison  between  2010-2011  season  and  2011-2012  season  for  birds.  Percentage  of 
usable  acquisitions  increased  in  the  current  year  even  if  the  total  number  of  recorded  events  is 
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Figure  3  -  Summary  of  midges  acquisitions  grouped  by  the  overall  mark  of  the  acquisition  and  the  size  of 
the  swarm. 


Figure  4  -  Male  (on  the  left  side)  and  female  (on  the  right  side)  of  Isocladius  sylvestris,  family 
Chironomidae,  subfamily  Orthocladiinae  tribe  Cricotopus. 
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Figure  5  -  Scheme  of  a  partial  temporal  sequence  of  two  objects  A  and  B  as  seen  by  one  camera, 
overlapping  in  the  image  for  three  frames.  The  old  algorithm  would  assign  only  one  dynamic  link  to  each 
detected  object,  therefore  the  points  of  occlusion  would  belong  to  only  one  uninterrupted  trajectory, 
while  the  second  recovered  trajectory  would  result  as  a  broken  one. 
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Figure  6  -  Scheme  showing  the  same  example  shown  in  Figure.  5;  Thanks  to  the  assignment  of  multiple 
links  per  object,  it  is  possible  to  recover  four  uninterrupted  trajectories,  of  which  two  are  real  and  two 
are  ghosts  with  hybrid  object  identities. 


Figure  7  -  Percolated  dynamic  links,  producing  the  possible  paths  of  a  few  birds  over  a  few  consecutive 
frames  in  time. 
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Figure  8  -  Scheme  of  the  global  optimization  strategy  applied  to  select  the  correct  paths  between  the 
four  possible  paths  extracted  in  the  example  of  Fig.  6. 
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Figure  10  -  Scheme  illustrating  the  recursive  approach. 
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Figure  11  -  2D-traces  reconstructed  in  the  image-space  of  each  of  the  three  cameras.  A  single  bird  of  the 
event  20111125  a2  e07  is  shown. 


Figure  12  -  3D-trajectory  of  the  same  bird  of  the  event  20111125_a2_e07  shown  in  Fig.  11. 
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Figure  13  -  3D-trajectories  of  the  entire  flock  (roughly  481  birds)  of  the  event  20111125_a2_e07  (file  too 
large  -  available  upon  request). 


Figure  14  -  Normalized  lengths  distribution  of  the  reconstructed  trajectories  for  five  bird  flock  events. 
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Figure  15  -  3D-trajectories  of  a  midge  swarm  event  (roughly  100  midges). 
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Table  1  -  Performance  of  the  implemented  algorithm  in  processing  11  field  acquisitions  of  flocking  birds. 


Section  3  -  Accomplishments  during  the  third  year  of  performance  (2012-2013)  of  the  group 
of  Co-PI  Cavagna  in  Rome: 


During  this  period  Cavagna  and  his  group 

•  Extended  the  experimental  data-taking  campaign  for  midges  to  collect  audio  and  video 
synchronous  data. 

•  Implemented  some  modifications  to  the  tracking  software  and  found  the  optimal  tracking 
parameters  for  midge  data. 

•  Ran  tracking  algorithm  on  new  flocking  and  swarming  events. 

•  Analyzed  the  data  and  developed  new  theoretical  approaches  and  numerical  models  of 
collective  animal  behavior. 

•  Wrote  and  submitted  the  results  of  the  work  in  high  profile  peer-reviewed  journals. 
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(1)  A.  Attanasi,  A.  Cavagna,  L.  Del  Gastello,  I.  Giardina,  T.S.  Grigera,  A.  Jelic,  S.  Melillo,  L. 
Parisi,  O.  Pohl,  E  Shen,  and  M.  Viale,  Superfluid  transport  of  information  in  turning  flocks  of 
starlings.  arXiv:  1303.7097  [cond-mat.stat-mech]  (2013).  Manuscript  submitted  to  Science. 

(2)  A.  Attanasi,  A.  Cavagna,  L.  Del  Gastello,  I.  Giardina,  A.  Jelic,  S.  Melillo,  L.  Parisi,  E.  Shen, 
E.  Silvestri,  M.  Viale,  Tracking  in  three  dimensions  via  multi-path  branching.  arXiv:  1305. 1495 
[q-bio.QM]  (2013).  Manuscript  submitted  to  Transaction  of  Pattern  Analysis  and  Machine 
Intelligence. 

(3)  A.  Cavagna  and  I.  Giardina,  Bird  flocks  as  condensed  matter  systems.  Annual  Review  of 
Condensed  Matter  Physics  5  (1),  2013 


Experiments  -  Midge  Swarms 

In  the  period  June  2012  -  May  2013,  a  new  experimental  campaign  took  place  to  collect  new 
field  data  of  swarming  midges.  The  aim  of  the  campaign  was  to  collect  perturbation/response 
data  of  swarms.  In  order  to  record  the  audio  track,  an  omni-directional  microphone  has  been  used 
together  with  a  digital  audio  recorder.  An  electronic  circuit  was  developed  to  transform  the 
manual  trigger  signal  used  to  start  the  video  acquisition  into  an  audio  tone,  and  feed  it  to  the 
audio  recorder.  In  this  way,  audio  and  video  tracks  were  synced.  Several  acquisitions  have  been 
collected  while  perturbing  the  midge  swarms  with  fixed-frequency  acoustic  pulses.  The 
frequency  range  from  390  Hz  to  480  Hz  has  been  investigated,  and  tones  have  been  played  with 
durations  equal  to  0.25  s  and  0.50  s.  The  most  significant  reaction  of  the  swann  has  been 
obtained  with  tones  at  480  Hz  and  0.5  s  long.  In  most  cases  the  typical  reaction  of  a  swann  to  a 
sound  input  is  revealed  by  a  sudden  expansion  of  the  swarm.  Swarm  data  under  noise 
perturbations  are  currently  under  investigation. 


Tracking 


Some  modifications  have  been  implemented  in  the  tracking  software  in  the  period  June  2012  - 
May  2013.  The  software  has  been  used  to  process  three  more  video  acquisitions  of  flocking 
birds,  and  eleven  video  acquisitions  of  swarming  midges.  This  work  allowed  us  to  find  the 
optimal  tuning  of  the  tracking  parameters  for  midge  data.  Large  midge  swarms  have  been 
processed,  up  to  swarms  of  700  individuals.  The  most  successful  tracking  elaborations  are 
summarized  in  Table  1  of  Publication  #2. 

Publication  #2 


Superfluid  Information  Transfer 

In  the  reporting  period  we  made  a  significant  discovery  in  collective  decision-making.  In 
biological  systems,  group  decision-making  requires  a  change  of  state  of  all  individuals,  with  the 
risk  of  cohesion  loss.  Therefore,  it  is  crucial  to  understand  the  main  factors  that  affect  the 
efficiency  of  information  transport  during  collective  changes  of  state. 

By  using  the  retrieved  3D  trajectories  of  individual  birds,  we  could  quantitatively  characterize 
the  formation  and  propagation  of  turns  across  starling  flocks.  We  discovered  that  the  information 
to  turn  propagates  linearly  in  time  with  negligible  damping,  in  contrast  with  current  theories  of 
collective  motion,  which  predict  a  diffusive  and  dissipative  transport  of  directional  information. 
We  formulated  a  novel  theory  that  recognizes  the  existence  of  a  conserved  spin  current  generated 
by  the  rotational  symmetry  of  the  system.  Our  theory  predicts  that  the  speed  of  propagation  of 
information  must  be  larger  the  stronger  the  flock's  orientational  order,  which  is  quantitatively 
confirmed  by  the  data.  Surprisingly,  our  novel  theory  is  mathematically  equivalent  to  that  of 
second  sound  propagation  in  superfluid  liquid  Helium,  which  is  a  compelling  demonstration  of 
the  powerful  consequences  of  symmetry  in  the  laws  regulating  natural  systems. 


Publication  #1 


Section  4  -  Accomplishments  during  the  fourth  year  of  performance  (2013-2014)  of  the 
group  of  Co-PI  Cavagna  in  Rome 

In  this  reporting  period  we  have:  collected  new  experimental  data  of  mosquito  swarms  in  the 
laboratory;  processed  and  analyzed  a  first  part  of  the  mosquito  data;  further  analyzed  the 
experimental  data  of  midge  swarms  in  the  field  collected  during  the  previous  years,  and 
developed  new  theoretical  approaches  and  numerical  models  of  collective  animal  behavior; 
validated  with  synthetic  data  visual  tracking  software;  wrote  and  submitted  results  in  high  profile 
peer-reviewed  journals. 
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Rossaro,  E.  Shen,  E.  Silvestri,  and  M.  Viale,  Finite-size  scaling  as  a  way  to  probe  near-criticality 
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Experiments  -  Mosquito  Swarms 

In  the  period  July  2013  -  December  2013  a  new  experimental  data-taking  campaign  took  place 
to  collect  new  3d  data  of  Anopheles  gambiae  mosquitoes  swarming  in  a  controlled  artificial 
environment. 

In  collaboration  with  the  Genomics,  Genetics  and  Biology  Pole  at  the  University  of  Perugia, 
Italy,  a  new  experiment  was  designed  and  set  up.  The  pole  offers  unique  facilities  for  mosquito 
population  studies,  including  several  large  swanning  cages  (5x3x2. 5  m,  LxWxH)  with  computer- 
controlled  temperature,  humidity,  illumination  intensity  and  illumination  frequency,  as  well  as  a 
large  insectary  for  mosquito  rearing.  Using  the  proper  temperature  and  humidity  conditions, 


together  with  the  simulated  daylight  cycle,  we  were  able  to  induce  a  fraction  of  the  mosquitoes 
introduced  inside  the  cage  to  swann  over  an  artifical  landmark  positioned  on  the  ground.  Swarms 
up  to  200  individuals  were  formed  and  recorded.  To  the  best  of  our  knowledge,  this  is  the  first 
time  that  large  mosquito  swarms  were  formed  in  an  artificial  environment. 

The  video  acquisition  setup  used  for  wild  swann  of  midges  during  the  previous  years  was 
adapted  to  this  specific  indoor  application  (camera  optics  and  supports).  A  powerfull  near- 
infrared  (NIR)  illumination  system  was  designed,  purchased,  and  installed  in  the  cage:  it 
provided  the  necessary  illumination  for  video  acquisitions,  without  dusturbing  with  visible  light 
the  insects  swarming  at  dusk.  NIR-absorbant  materials  were  tested  and  used  as  imaging 
background.  The  video  system  was  calibrated  using  the  same  NIR  light,  and  the  3d  stereometric 
reconstruction  was  accurately  tested. 

Experiments  were  run  on  a  daily  basis  during  the  artificial  sunset  phase  lasting  roughly  40 
minutes.  In  total,  105  video  acquisitions  (16  s  each)  of  mosquito  swarming  events  were  taken. 

Data  analysis  -  Mosquito  Swarms 

In  the  period  January  2014  -  May  2014  a  first  part  of  the  mosquito  swarms  data  (roughly  20%  of 
the  acquisitions)  was  processed  using  our  tracking  software.  Data  analysis  of  the  processed 
experimental  events  was  started  with  particular  emphasis  on  the  connected  velocity  correlations, 
and  on  the  order  parameters.  Mosquito  data  processing  and  analysis  are  ongoing. 

Data  analysis  -  Midge  Swarms 

The  most  prominent  trait  of  collective  behavior  is  the  emergence  of  collective  order.  Swarms  of 
midges,  though,  are  not  collectively  ordered.  Moreover,  swanns  are  generally 
found  in  proximity  of  a  visual  marker,  such  as  a  water  puddle  or  a  white  feature  over  a  black 
background.  The  presence  of  the  landmark,  together  with  the  absence  of  collective  order,  makes 
the  debate  on  whether  swarms  display  truly  collective  behavior  very  intriguing. 

In  the  reporting  period,  we  analyzed  21  swanning  events  collected  in  the  field.  For  each 
swarming  event,  we  reconstructed  the  3d  trajectories  of  each  midge  using  our  tracking  algorithm. 
We  used  these  trajectories  to  retrieve  position  and  velocity  of  each  individual  at  each  instant  of 
time,  and  we  investigated  how  much  a  change  in  the  direction  of  one  midge  affects  that  of  other 
individuals. 

We  discovered  that,  despite  the  lack  of  collective  order,  swarms  exhibit  non- trivial  correlations 
of  the  direction  of  motion,  totally  incompatible  with  models  on  non-interacting  particles.  This 
proves  that  a  true  collective  behavior  emerges  among  midges  while  swarming,  and  that  the  true 
hallmark  for  collective  behavior  is  not  collective  order  but  a  strong  and  non-trivial  correlation. 
We  found  that  correlation  increases  sharply  with  the  swarm  density,  indicating  that  the 
interaction  between  midges  is  based  on  a  metric  perception  mechanism. 

By  means  of  numerical  simulations  we  demonstrated  that  such  growing  correlation  is  typical  of  a 
system  close  to  an  ordering  transition.  Moreover,  this  indicates  that  midges  adjust  their  mutual 


distance  according  to  the  size  of  the  swarm  in  order  to  be  maximally  correlated.  As  a 
consequence,  the  correlation  length  scales  with  the  system’  size  and  swarms  exhibit  a  near- 
maximal  degree  of  correlation  at  all  sizes. 

These  results  are  summarized  in  two  different  papers,  Publications  #1  and  #4. 

Tracking 

In  the  reporting  period,  several  sets  of  synthetic  data  were  produced  in  order  to  validate  the 
tracking  algorithm  and  software.  Computer  simulations  of  bird  flocks  were  run,  and  the 
simulated  3d  trajectories  were  used  to  retrieve  synthetic  image  data.  The  quality  of  the  output 
trajectories  was  measured  processing  these  datasets  with  known  ground-truth,  prooving  the 
excellent  performances  of  the  software. 

These  results  were  added  to  the  first  manuscript  describing  our  tracking  software,  and  the 
manuscript  was  fully  rewritten.  This  version  also  benefits  from  the  collaboration  with  a  new 
author,  Fabio  Pellacini,  professor  of  Computer  Science.  The  new  manuscript  was  submitted  as 
Publication  #3. 


